Nonrigid Object Segmentation and Occlusion Detection in Image Sequences

We address the problem of nonrigid object segmentation in image sequences in the presence of occlusions. The proposed variational segmentation method is based on a region-based active contour of the Chan-Vese model augmented with a frame-to-frame interaction term as a shape prior. The interaction term is constructed to be pose-invariant by minimizing over a group of transformations and to allow moderate deformation in the shape of the contour. The segmentation method is then coupled with a novel variational contour matching formulation between two consecutive contours which gives a mapping of the intensities from the interior of the previous contour to the next. With this information occlusions can be detected and located using deviations from predicted intensities and the missing intensities in the occluded regions can be reconstructed. After reconstructing the occluded regions in the novel image, the segmentation can then be improved. Experimental results on synthetic and real image sequences are shown.